Progressive Ladder Networks for Semi-Supervised Transfer Learning

Semi-supervised learning has achieved remarkable success in the past few years at harnessing the power of unlabeled data and tackling domains where few labeled data examples exist. We test the hypothesis that deep semisupervised architectures learn general representations. We combine two well-known techniques for semi-supervised and transfer learning, ladder networks and progressive neural networks, to create the progressive ladder network, a framework for transferring from semi-supervised representations to supervised representations. We use this framework for domain adaptation in the digit recognition domain, from the MNIST handwritten digits dataset to the Street View House Numbers (SVHN) dataset. Our experiments show that semi-supervised representations are more transferable than supervised learning representations. We find that using our progressive ladder network architecture on simple fully connected neural networks trained on SVHN can yield an increase in test accuracy of about 40%, from around 35% to 77.7%.